Experimenting with the Path Ranking Algorithm

The Path Ranking Algorithm (Lao and Cohen, 2010) is a general technique for performing link prediction in a graph. PRA has mainly been used for knowledge base completion (Lao et al., 2011; Gardner et al., 2013; Gardner et al., 2014), though the technique is applicable to any kind of link prediction task. To learn a prediction model for a particular edge type in a graph, PRA finds sequences of edge types (or paths) that frequently connect nodes that are instances of the edge type being predicted. PRA then uses those path types as features in a logistic regression model to infer missing edges in the graph. In this class project, we performed three separate experiments relating to different aspects of PRA: improving the efficiency of the algorithm, exploring the use of multiple languages in a knowledge base completion task, and using PRA-style features in sentencelevel prediction models. The first experiment looks at improving the efficiency and performance of link prediction in graphs by removing unnecessary steps from PRA. We introduce a simple technique that extracts features from the subgraph centered around a pair of nodes in the graph, and show that this method is an order of magnitude faster than PRA while giving significantly better performance. Additionally, this new model is more expressive than PRA, as it can handle arbitrary features extracted from the subgraphs, instead of only the relation sequences connecting the node pair. The new feature types we experimented with did not generally lead to better predictions, though further feature engineering may yield additional performance improvements. The second experiment we did with PRA extends recent work that performs knowledge base completion using a large parsed English corpus in conjunction with random walks over a knowledge base (Gardner et al., 2013; Gardner et al., 2014). This prior work showed significant performance gains when using the corpus along with the knowledge base, and even further gains by using abstract representations of the textual relations extracted from the corpus. In this experiment, we attempt to extend these results to a multilingual setting, with textual relations extracted from 10 different languages. We discuss the challenges that arise when dealing with data in languages for which parsers and entity linkers are not readily available, and show that previous techniques for obtaining abstract relation representations do not work in this setting. The final experiment takes a step towards a longstanding goal in artificial intelligence research: using a large collection of background knowledge to improve natural language understanding. We present a new technique for incorporating information from a knowledge base into sentence-level prediction tasks, and demonstrate its usefulness in one task in particular: relation extraction. We show that adding PRAstyle features generated from Freebase to an off-theshelf relation extraction model significantly improves its performance. This simple and general technique also outperforms prior work that learns knowledge base embeddings to improve prediction performance on the same task. In the remainder of this paper, we first give a brief introduction to the path ranking algorithm. Then we discuss each experiment in turn, with each section introducing the new methods, describing related work, and presenting experimental results.